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1.0  Executive  Summary 

1.1  Background 

This  report  represents  a  test  and  evaluation  (T&E)  of  the  Field  Medieal 
Surveillance  System  (FMSS),  a  product  of  the  Field  Medical  Technologies 
program  of  the  Naval  Health  Research  Center  (NHRC). 

1.2  Method 

Several  data  sources  were  used  to  conduct  this  study,  including; 

•  A  usability  survey  of  eight  users  experienced  with  medical  software  who 
fdled  out  structured  survey  questions  and  provided  comments 

•  A  functionality  survey  that  systematically  tested  35  basic  operational 
claims  drawn  from  the  FMSS  user’s  manual 

•  Validation  testing  by  three  technical  consultants  who  evaluated  the  35 
operations  above  and  provided  structured  survey  ratings  and  feedback 
from  the  standpoint  of  users  with  medical  knowledge  and  deployment 
experience 

•  A  review  of  available  literature  and  Web-based  resources  on  competitive 
software  packages 

1.3  Results 

Results  of  the  study  include  the  following: 

•  FMSS  met  nearly  all  of  the  claims  advertised  by  the  developers  associated 
with  creating  and  maintaining  a  patient  database,  generating  disease 
surveillance  graphs  and  reports,  and  providing  current  medical  references. 

•  Survey  results  and  user  reviews  indicated  that  FMSS  was  appropriate  for 
use  as  a  surveillance  tool  for  deployed  Environmental  Health  Officers 
(EHOs)  and  Preventive  Medicine  Officers  (PMOs). 

•  Fimitations  noted  for  FMSS  included  potential  lack  of  compatibility  with 
the  Medical  Data  Surveillance  System  (MDSS)  and  other  medical 
surveillance  tools,  mainly  related  to  the  exportability  of  the  International 
Classification  of  Diseases,  9th  Revision  (ICD-9)  codes. 

•  Other  FMSS  issues  touched  on  in  this  study  include  inconsistencies 
between  the  program  and  the  user’s  manual,  lack  of  flexibility  to  modify 
graphs  and  reports,  and  unclear  error  messages. 
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1.4  Conclusions 

FMSS  performed  satisfaetorily  in  meeting  advertised  elaims.  Developers  should 
address  users’  eoneerns  about  ineonsisteneies  between  the  eurrent  version  of  the 
program  and  the  user’s  manual.  Most  importantly,  time  series  and  ineidenee  rate 
graphs  should  funetion  as  indieated.  Another  area  of  eoneem  is  the  eompatibility 
of  FMSS  with  other  medieal  applieations  that  require  the  use  of  ICD-9  eodes.  It  is 
not  elear  from  the  program  or  the  user’s  manual  whether  eodes  are  attached  to 
diagnoses  made  through  all  available  FMSS  options.  Finally,  the  addition  of  other 
required  reports  such  as  disease  and  nonbattle  injury  would  enhance  the 
usefulness  of  the  program. 
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2.0  Introduction 

The  Field  Medieal  Teehnologies  program  at  NHRC  is  working  to  optimize  and  integrate 
systems  for  field  use  at  various  levels  of  eare,  ineluding  remote  environments.  To  do  so, 
it  is  neeessary  to  evaluate  how  available  software  might  serve  this  purpose  and/or 
augment  existing  NHRC  produets  sueh  as  MDSS.  It  is  also  neeessary  to  provide 
eomparative  analyses  of  systems  designed  to  serve  similar  funetions,  sueh  as  the  Field 
Medieal  Surveillanee  System  (FMSS). 

2.1  Background 

FMSS  is  a  produet  of  NHRC  and  the  FMT  program  that  has  been  fielded  for  use 
at  the  request  of  a  limited  number  of  programs  outside  NHRC  (e.g.,  Ventura 
County  Health  Department,  U.S.  military  Environmental  and  Preventive  Medieine 
Units  ).  The  developers  also  have  reeeived  inquiries  about  the  produet  from  the 
Federal  Emergeney  Management  Ageney,  the  Bureau  of  Indian  Affairs,  and  the 
Polish  Navy.  Informal  feedbaek  to  developers  from  these  agencies  has  been 
positive,  but  there  has  been  no  formal  evaluation  report  on  the  EMSS  product. 

The  present  work  was  conducted  by  a  research  team  at  NHRC  separate  from  the 
EMSS  developer  to  test  the  baseline  functioning  of  the  product.  The  developers 
are  presently  working  on  a  Windows  CE  version  of  basic  EMSS  functions  for  use 
on  personal  digital  assistants  (Martin  White,  personal  communication,  August  12, 
2003). 

2.2  Objectives 

The  objectives  of  this  test  and  evaluation  (T&E)  study  include  the  following: 

•  To  conduct  T&E  of  EMSS  software  Version  1.0,  provided  July  2003,  to 
document  a  baseline  of  its  functioning  relative  to  the  developers’ 
advertised  objectives  and  specific  claims 

•  To  provide  feedback  and  recommendations  to  developers  for  system 
improvement  for  usability  and  functioning 

2.3  Description  of  FMSS 

The  EMSS  user’s  manual  describes  EMSS  as  intended  for  use  at  the  first  echelons 
of  care  to  help  detect  emerging  health  problems  that  might  occur  during  foreign 
deployments  or  conflicts.  ’  EMSS  is  intended  for  installation  and  use  on  a  laptop 
computer.  It  is  a  medical  information  and  analysis  system  that  incorporates  new 
patient  encounters,  provider  information,  and  medical  reference  information. 

EMSS  is  intended  for  use  by  Environmental  Health  Officers  (EHOs)  and 
Preventive  Medicine  Officers  (PMOs)  during  deployments.  Its  goal  is  to  minimize 
the  impact  of  disease  on  deployed  forces  by  providing  EHOs  and  PMOs  with 
timely  access  to  summary  and  analysis  of  medical  information.  The  development 
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of  FMSS  followed  Department  of  Defense  (DoD)  requirements  for  deployment 
medieal  surveillanee.^ 

The  user’s  manual  states  that  FMSS  ean  help  determine  ineidenee  rates;  project 
short-term  trends;  profile  the  characteristics  of  affected  populations  by  person, 
time,  or  place;  track  modes  of  disease  transmission;  and  generate  various  graphs 
and  reports.  FMSS  also  incorporates  the  Global  Infectious  Disease  and 
Epidemiology  Network  (GIDEON),  using  an  interface  specific  to  military 
requirements.  This  is  a  well-known  database  for  infectious  and  parasitic  diseases 
from  over  205  countries.  It  is  designed  to  help  diagnose  most  of  the  world’s 
infectious  diseases  based  on  signs,  symptoms,  and  laboratory  findings  that  are 
entered  for  patients.  EMSS  also  has  a  comprehensive  list  of  injuries,  noninfectious 
diseases,  and  mental  illnesses  that  can  be  selected  without  the  need  for  detailed 
symptomology. 

EMSS  also  provides  online  medical  references  such  as  the  Control  of 
Communicable  Diseases  Manual  (CCDM),  Tri-Service  Reportable  Events 
Guidelines  and  Case  Definitions  (TSRE),  and  select  reports  from  the  Armed 
Eorces  Medical  Intelligence  Center’s  (AEMIC)  Medical  Environmental  Disease 
Intelligence  and  Countermeasures  (MEDIC)  CD-ROM. 
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3.0  Market  Research 

A  broad  review  of  military  medical  information  systems  that  may  overlap  with  FMSS 
functions  was  conducted  to  provide  a  context  for  the  potential  applications  of  FMSS  and 
determine  if  it  has  unique  functions  or  technical  capabilities. 

3.1  SAMS 

The  SNAP  Automated  Medical  System  (SAMS)  is  a  multi-user  administrative 
management  tool.  SAMS  is  in  use  by  more  than  1400  sites,  including  Navy 
operational  commands  and  Marine  Corps  medical  units,  clinics,  and  medical 
treatment  facilities.^  SAMS  documents  and  builds  reports  for  medical  encounters, 
environmental  health,  all-hands  medical  training,  and  laboratory  results;  it  also 
tracks  immunizations.  SAMS  does  not  include  embedded  medical  training.  SAMS 
was  evaluated  by  MTS  Technologies,  Inc.,  during  field-testing  at  Echelon  1  and  2 
medical  treatment  facilities  (MTFs)  in  support  of  the  Cobra  Gold  2002  exercise  in 
Thailand.  Currently,  SAMS  is  fielded  in  the  Iraq  theater  and  is  providing  patient 
encounter  information  to  the  MDSS  server. 

3.2  CHCS 

The  Composite  Health  Care  System  (CHCS)  provides  automated  medical 
information  support  to  all  MTFs  worldwide.  CHCS  captures  data  on  patient 
registration,  admission,  disposition,  and  transfer;  inpatient  activity;  outpatient 
administration;  appointment  scheduling;  laboratory  drug/laboratory  test 
interaction;  quality  assurance;  radiology;  clinical  dietetic  administration;  and 
pharmacy^.  The  system  conducts  results  reporting  and  order  entry  as  well  as  ad 
hoc  reporting.  Through  CHCS  connectivity,  users  can  access  other  medical 
facilities  and  receive  specific  patient  information. 

The  release  of  CHCS-II  expands  the  system  to  include  support  of  general 
dentistry  and  optometry.  It  also  includes  interface  capability  with  the  Defense 
Enrollment  Eligibility  Reporting  System,  the  Third  Party  Outpatient  Collection 
System  (TPOCS),  and  the  Pharmacy  Data  Transaction  Service.  CHCS  utilizes 
ICD-9  codes  in  processing  data.  While  EMSS  appears  to  be  more  mobile  and  can 
be  used  on  a  network  or  a  stand-alone  computer,  CHCS  is  a  network-driven 
program  that  excels  when  connected  to  numerous  facilities.  Currently,  CHCS  is 
integrated  with  MDSS,  providing  patient  data  to  be  analyzed  for  detecting  and 
reporting  changes  in  the  state  of  the  health  of  a  population. 
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3.3  GEMS 

The  Global  Expeditionary  Medical  System  (GEMS)  is  currently  fielded  by  the  EIS 
Air  Eorce.  It  is  made  up  of  three  different  Internet-based  software  applications: 
the  Patient  Module,  the  Theater  Epidemiology  Module,  and  the  Theater 
Occupational  Module.  Together,  these  modules  can  detect  trends  in  symptoms 
and  diagnosis  with  military  patients  and  can  help  physicians  determine  if  troops 
have  been  exposed  to  biological  warfare  agents.^  GEMS  performs  an  analysis  and 
graphically  displays  the  deployed  force’s  collective  health  and  readiness.  In 
addition,  it  records  and  tracks  data  to  detect  illness  caused  by  workplace,  military, 
or  natural  causes.  Its  software  functions  especially  well  on  a  laptop  or  handheld 
computer  that  medics  can  use  to  record  patient  information  in  the  field  and 
transmit  it  for  detailed  analysis.  It  incorporates  the  Special  Operations  Medical 
Handbook,  a  field  operator’s  guide,  the  Medical  NBC  Battlebook,  and  Mosby 
RX/TX  instructions  for  the  user.  The  US  Government  owns  the  GEMS  source 
code. 

3.4  SOMDS 

The  Special  Operations  Medical  Diagnostic  System  (SOMDS)  is  a  computerized 
medical  diagnostic  and  treatment  system  for  the  support  of  Special  Operations 
Eorces  (SOE)  medical  personnel.  SOMDS  is  designed  to  be  used  for  the  medical 
assessment,  diagnosis,  and  treatment  of  patients  in  remote  areas  that  may  be  far 
removed  from  routine  medical  facilities.  The  objective  of  SOMDS  is  to  enhance 
the  capabilities  of  SOE  medical  personnel  to  assess  and  manage  medical 
situations  and  casualties  in  the  field.  This  objective  is  realized  through  the 
development  and  implementation  of  clinical  guidelines  that  bring  medical 
information  to  the  point  of  need  through: 

•  An  embedded  expert  system  that  provides  a  comprehensive  medical 
record  and  computer-assisted  diagnosis  and  treatment  information  at  the 
point  of  need  based  on  evidence-based  practice  guidelines.  An  embedded 
interactive  training  mode  that  provides  remotely  deployed  SOE  medical 
personnel  with  the  means  to  sustain  medical  proficiency  and  develop  new 
skills  by  accessing  the  expert  system  and  medical  references  during  idle 
periods. 

•  An  information  system  that  provides  teleconsultation  and  store-and- 
forward  capabilities  to  remote  areas. 
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3.5  MDSS 

The  Medical  Data  Surveillance  System  is  a  Web-based  information  system  that 
analyzes  ICD-9  codes  and  allows  access  to  real-time  medical  threat  assessment 
for  deployed  forces.  The  software  facilitates  response  to  medical  threats  by 
providing  the  PMO,  epidemiologist,  or  Commander-in-Chief  (CINC)  Surgeon 
with  automated  tools  to  assist  in  the  process  of  investigating,  identifying,  and 
reporting  significant  medical  events. 

MDSS  automatically  searches  patient  ICD-9  codes  for  trends  based  on 
epidemiological  clues  identified  by  the  Centers  for  Disease  Control  and 
Prevention.  The  system  is  intended  to  provide  epidemiologists  with  tools  for  early 
detection  of  disease  outbreaks  and  chemical  attacks.  It  also  promises  to  give 
medical  command  and  control  the  ability  to  integrate  patient  data  from  widely 
dispersed  forces  for  mission  planning. 

The  key  feature  of  MDSS  is  advanced  dynamic  change  point  detection  analysis, 
which  is  designed  to  allow  early  detection  of  illness  trends  and  disease  outbreaks. 
MDSS  analyzes  ICD-9  codes  using  a  set  of  dynamic  change  point  and  signal 
detection  algorithms  to  identify  the  start  and  end  points  of  medical  events,  trends, 
and  shifts  within  routinely  collected  data.  The  system  is  able  to  identify  incidence 
spikes  using  relatively  small  data  sets. 

MDSS  enables  epidemiologists  to  set  and  adjust  baseline  and  threshold  values, 
detect  shifts  and  trends  within  data,  and  reconstruct  the  signal  to  show  the  form  of 
the  underlying  event.  MDSS  automatically  alerts  medical  users  of  abnormalities 
and  provides  tools  so  that  users  can  investigate  the  nature  and  source  of  medical 
events.  The  system  can  calculate  baselines  based  on  a  week’s  worth  of  data  or  less 
from  an  MTF  and  provides  early  warning  of  trends  to  alert  the  medical  command 
of  possible  disease  outbreaks  before  they  become  epidemic. 
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4.0  Method 

The  present  study  focuses  on  the  validation  of  the  following  claims  for  basic  FMSS 

functions,  set  forth  in  the  FMSS  user’s  manual: 

•  Determine  incidence  rates  for  disease  and  injury 

•  Project  short-term  trends 

•  Profde  by  person,  time  period,  or  place 

•  Track  modes  of  disease  transmission 

•  Generate  graphs  and  reports 

•  Provide  access  to  medical  references 

Validation  of  these  claims  was  conducted  via  three  steps: 

•  Functionality  testing 

•  A  usability  review 

•  A  review  by  technical  consultants 

4.1  Functionality  Testing 

Two  members  of  the  T&E  team  independently  assessed  each  of  these  claims 
during  validation  testing.  Testers  prepared  a  list  of  35  functional  requirements 
extracted  from  the  FMSS  user’s  manual.  Each  function  or  group  of  functions  was 
tested  and  then  assigned  a  pass  or  fail  decision.  (Testers  were  in  agreement  on  all 
cases  except  one.  In  this  case,  other  members  of  the  T&E  team  provided  input  to 
resolve  the  decision,  which  will  be  discussed  below.) 

4.2  Usability  Review 

Eight  members  of  the  T&E  team  who  were  experienced  with  military  medical 
software  applications  reviewed  the  functions  of  EMSS  while  operating  the 
installed  application  for  a  period  of  approximately  2  to  3  hours.  The  T&E 
personnel  then  completed  the  Elsability  Survey  (see  Appendix  B).  This  survey 
was  originally  designed  to  assess  commercial-off-the-shelf  products  based  on 
current  human  factors  principles.'^  It  was  adapted  for  EMSS  for  the  current 
analyses.  The  survey  uses  Eikert  scale  ratings,  accompanied  by  space  for  brief 
explanatory  comments  and  elaborations. 

4.3  Technical  Consultant  Review 

Three  technical  consultants  employed  by  MTS  Technologies,  Inc.,  evaluated 
EMSS  from  the  standpoint  of  their  experience  as  military  medical  personnel  in 
deployed  settings.  During  testing,  T&E  team  members  presented  the  technical 
consultants  with  each  of  the  35  requirements  described  above  and  asked  them  to 
attempt  to  complete  the  specified  functions.  Then,  each  technical  consultant  filled 
out  a  brief  background  survey  and  the  Usability  Survey  described  above,  which 
asked  them  to  rate  overall  usability  and  to  comment  on  the  strengths  and 
weaknesses  of  the  product.  The  User  Survey  is  included  as  Appendix  C. 
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5.0  Results 

5.1  Functionality  Testing 

Using  the  FMSS  manual,  T&E  team  members  generated  a  list  of  basie  functions 
to  be  tested  (see  Appendix  D).  These  were  then  used  to  determine  whether  FMSS 
fulfilled  its  major  claims,  which  were; 

•  Determine  incidence  rates  for  disease  and  injury 

•  Project  short-term  trends 

•  Profile  by  person,  time  period,  or  place 

•  Track  modes  of  disease  transmission 

•  Generate  graphs  and  reports 

•  Provide  access  to  medical  references 

In  order  to  generate  the  reports  and  analyses  that  fulfill  its  basic  functions,  FMSS 
must  be  able  to  create  a  database.  Users  tested  the  ability  of  the  program  to  allow 
the  creation  of  a  new  database  or  the  use  of  an  existing  one.  FMSS  performed  to 
specifications  in  creating  a  new  unit  and  new  patient  demographics  and  in  editing 
existing  ones.  Users  were  also  able  to  generate  or  edit  diagnoses  from  the  various 
options  available:  signs  and  symptoms,  disease  profdes,  area-specific  disease 
data,  medical  references,  clinical  studies,  and  consultations.  Then,  using  the 
database  provided,  they  tested  the  functionality  claims  regarding  the  generation  of 
reports  and  analyses. 

5.1.1  Incidence  Rates  for  Disease  and  Injury 

Testers  were  able  to  select  and  view  the  Incidence  Rate  Text  Report.  They 
were  not  able  to  create  or  view  a  Time  Series  or  Incidence  Rate  Series 
graph  created  from  selected  dates  or  diseases,  which  are  listed  as  options 
in  the  user’s  manual. 

5.1.2  Projection  of  Short-Term  Trends 

Testers  were  able  to  view  a  trend  analysis  report.  However,  this  option  is 
not  listed  or  described  in  the  user’s  guide. 

5.1.3  Profile  by  Person,  Time  Period,  or  Place 

Testers  were  in  disagreement  over  whether  this  option  functioned  as 
advertised.  Testers  were  able  to  generate  Surveillance  graphs,  which 
profile  diseases  by  variables  including  agent,  dates,  and  patient 
demographics.  However,  the  user’s  manual  suggests  that  the  text  edit  box 
will  allow  variables  to  be  selected  specifically  to  be  the  X  or  Y  axis  of  the 
graph,  and  this  was  not  possible. 


September  1 0,  2003 


5-1 


Test  and  Evaluation  Report  for  FMSS 


Version  1 


5.1.4  Track  Modes  of  Disease  Transmission 

The  Surveillance  Graph  option  allows  users  to  view  selected  diseases  by 
transmission  characteristics.  These  include  reservoir,  vector,  and  vehicle 
variables.  Users  were  able  to  select  these  variables  to  be  graphed. 

5.1.5  Generate  Graphs  and  Reports 

Testers  were  able  to  generate  daily  logs,  patient  histories,  reportable 
conditions,  classification  reports,  trend  analysis  reports,  and  a  text  report 
on  incidence  rates.  Testers  were  also  able  to  generate  Surveillance  graphs 
with  the  limitations  mentioned  above  (see  Section  5.1.3).  However,  testers 
were  not  able  to  create  Time  Series  or  Incidence  Rate  Series  graphs  as 
listed  in  the  user’s  manual.  The  rates  in  the  graphs  are  expressed  as  “per 
100-person  days.”  Most  epidemiological  reports  express  rates  in  terms  of 
patients  seen  (e.g.,  number  of  cases  per  1000  patients). 

5.1.6  Provide  Access  to  Medical  References 

Users  were  able  to  view  the  CCDM,  MEDIC,  and  TSRE  resources,  as 
described  in  the  user’s  manual.  They  were  also  able  to  create  diagnoses  as 
specified  using  GIDEON,  the  ICD-9  disease  list,  signs  and  symptoms  for 
diseases  by  country  of  disease  acquisition,  and  disease  profiles  by  agent, 
vector,  reservoir,  or  vehicle.  One  limitation  noted  was  that  the  ICD-9 
reference  did  not  list  diseases  and  injuries  according  to  their  codes. 

5.1.7  Additional  Functions 

During  testing,  users  were  able  to  view  and  edit  the  patient  database  as  a 
spreadsheet.  This  option  is  not  listed  or  described  in  the  user’s  manual.  Eor 
this  reason,  the  features  of  this  function  were  not  evaluated  in  detail. 
However,  a  preliminary  assessment  found  some  potential  problems  with 
the  spreadsheet  (see  Appendix  E  ). 

5.2  Usability  Review 

Eight  members  of  the  T&E  team  used  EMSS  and  its  user’s  manual  for  a  total  of 
approximately  2  to  3  hours.  Then,  they  filled  out  a  survey,  which  rated  EMSS  on 
15  different  aspects  related  to  general  usability,  including  availability  and  clarity 
of  help,  appropriateness  of  language,  and  functions  for  the  target  user  and  overall 
organization.  Each  item  was  rated  on  a  5 -point  Eikert  scale  where  5  =  strongly 
agree,  1  =  strongly  disagree,  and  3  =  undecided.  The  results  of  this  survey  appear 
in  Table  1. 
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Table  1.  Usability  Review  Results 


Item  Summary 

Mean 

SD 

Uses  simple  language 

4.25 

0.46 

Language  familiar  to  target  user 

4.12 

0.83 

Requires  minimal  memory  load 

4.00 

0.76 

Consistent  use  of  terms  and  format 

3.75 

0.71 

Provides  informative  feedback  on  actions 

3.00 

0.93 

Navigational  functions  are  easy  to  find 

3.37 

0.92 

Shortcuts  available  to  experienced  users 

2.62 

0.74 

Error  messages  clear  and  constructive 

3.00 

0.93 

Design  minimizes  errors 

3.62 

0.74 

Provides  help  if  necessary 

2.87 

0.83 

Graphic  design  is  simple  and  intuitive 

3.75 

1.39 

Offers  flexibility/unwanted  steps  not  necessary 

2.62 

1.19 

Easy  to  learn  basic  functions 

4.25 

0.46 

Basic  functions  are  organized  reasonably 

4.00 

0.53 

Can  undo  or  redo  actions 

3.37 

1.51 

The  results  of  this  survey  suggest  that  users  were  in  substantial  agreement  that 
FMSS  was  reasonably  easy  to  learn,  targeted  appropriately  to  its  user,  and  well- 
organized  and  consistent  in  the  presentation  of  its  basic  functions.  Users  were 
undecided  as  to  whether  the  error  messages,  when  they  did  occur,  were  useful, 
and  whether  help  was  readily  available.  Help  provided  by  FMSS  was  not  linked  to 
functions,  but  the  full  manual  was  available  for  viewing  as  a  pdf  document.  There 
were  also  several  inconsistencies  between  the  user’s  manual  and  the  program 
itself.  The  most  negative  ratings  were  given  in  regard  to  the  availability  of 
shortcuts,  flexibility,  and  the  necessity  of  completing  unwanted  steps.  Users 
specifically  mentioned  that  graphs  were  not  readily  modifiable,  and  that  the 
process  of  creating  a  new  database  did  not  flow  from  one  step  to  the  next.  Several 
users  also  commented  on  the  default  “blank  screen”  format.  An  alternative  format 
presented  the  patient  database  as  a  spreadsheet,  but  this  was  not  specifically 
mentioned  in  the  manual  (see  section  5.1.7). 
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5.3  Technical  Consultants  Review 
5.3.1  Survey  Ratings 

Three  technieal  consultants  responded  to  six  survey  statements  on  a 
Likert-type  scale  of  agreement.  Responses  were  coded  from  strongly 
disagree  (1)  to  strongly  agree  (5).  Questions  and  a  summary  of  responses 
are  as  follows: 

•  The  medical  information  provided  through  FMSS  was  useful 
(M  =  3.33,  SD  =  0.58).  The  technical  consultants  indicated  that 
FMSS  provided  useful  information  within  its  limits,  but  that  both 
the  reference  material  and  the  reporting  options  were  not 
comprehensive. 

•  The  medical  information  provided  through  FMSS  was  easy  to  use 
(M  =  3.00,  SD  =  1.73).  Two  participants  rated  information  as  easy 
to  use.  The  remaining  technical  consultant  said  that  the  graphical 
presentation  of  the  data  was  confusing,  and  data  presentation 
options  were  not  sufficiently  related  to  stated  Navy  medical 
directives. 

•  The  medical  information  FMSS  provided  was  presented  in  a  useful 
format  (M  =  3.00,  SD  =  1.00).  This  rating  reflected  similar  views 
to  the  previous  question.  However,  only  one  technical  consultant 
agreed  unequivocally  that  the  format  was  useful.  One  was  neutral, 
and  the  remaining  technical  consultant  reiterated  that  the 
presentation  of  data  was  confusing. 

•  The  quality  of  the  medical  information  provided  by  FMSS  is  better 
than  that  provided  by  previous  reporting  methods  (M  =  2.67,  SD  = 
1.53).  Two  technical  consultants  rated  FMSS  as  equal  to  or  better 
than  other  electronic  methods,  and  one  noted  that  electronic 
methods  in  general  have  the  advantage  over  paper-and-pencil 
options.  However,  the  third  technical  consultant  gave  FMSS  an 
unfavorable  rating  compared  with  the  medical  information  and 
analytical  support  provided  by  MDSS,  GEMS,  and  the  Electronic 
Surveillance  System  for  the  Early  Notification  of  Community- 
based  Epidemics  (see  Section  6.0). 

•  FMSS  was  flexible  enough  to  meet  my  needs.  1  could  set  it  up  to  do 
what  1  wanted  it  to  do  efficiently  (M  =  2.00,  SD  =  0.00). 

Three  technical  consultants  agreed  that  FMSS  did  not  offer  much 
flexibility  in  customizing  reports.  Specific  issues  mentioned 
included  the  lack  of  an  option  for  a  date  query  in  the  trend  analysis, 
and  limitations  on  the  ability  to  investigate  signs  and  symptoms. 

•  FMSS  would  help  me  do  my  job  (M  =  3.33,  SD  =  0.58).  One 
technical  consultant  gave  a  favorable  “overall”  rating  to  this 
statement.  The  remaining  technical  consultants  were  neutral,  with 
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one  citing  problems  with  the  design  restrictions  mentioned  in 
previous  items. 
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5.3.2  Open-Ended  Items 

The  remaining  six  survey  items  gave  teehnieal  eonsultants  the  opportunity 

to  give  open-ended  eomments  in  response  to  questions  about  FMSS.  Their 

responses  are  listed  below. 

The  most  useful  thing  about  FMSS  for  my  job  would  be: 

•  Presenting  disease  trends,  alerts,  and  risks  to  senior  personnel. 

•  The  aecess  to  diseases  for  speeific  regions. 

•  Surveillance  reports  could  be  helpful  once  improved  and  more 
dynamic. 

The  least  useful  thing  about  FMSS  for  my  job  was: 

•  N/A.  All  aspects  were  applicable. 

•  Stand-alone  program  that  needs  to  be  incorporated  into  one 
standard  system  for  medical.  May  duplicate  workload  as  it  stands 
now. 

•  Having  to  enter  data. 

If  I  could  add  one  thing  to  FMSS,  it  would  be: 

•  I  would  add  a  design  option  using  contributions  or 
recommendations  of  end  users  and  not  superiors,  higher-ups,  or 
personnel  not  directly  involved  or  well-experienced  in  subject 
matter. 

•  Additional  information  from  the  Merck  Manual. 

•  Data  query  analysis. 

What  features  of  FMSS  did  you  like? 

•  Graphs  and  tables. 

•  Disease  access  to  specific  regions. 

•  Surveillance  comparisons.  Would  like  to  see  it  compare  or  analyze 
more  than  two  fields. 

What  features  of  FMSS  did  you  dislike? 

•  Very  little.  Recommend  proofreading  of  verification  of  terms, 
syntax,  verbiage,  etc.,  in  relation  to  specific  subjects,  i.e., 
mammals:  man,  horse,  goat,  deer,  pig,  dog,  etc. 

•  Can  only  open  one  form  at  a  time.  Data  would  have  to  be  entered 
after  patient  was  seen  since  you  cannot  print  specific  medical 
encounters. 

•  General  graphs. 
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Where  would  you  consider  FMSS  a  best  fit  in  the  military  medical  arena? 

•  In  the  upper  eehelon  preventive  medieine  area. 

•  As  the  system  is  now,  it  would  fit  into  a  preventive  medieine  unit. 
With  additional  features  sueh  as  standard  medieal  forms,  it  eould 
be  used  in  any  operational  medieal  unit. 

•  Military  is  looking  for  programs  with  minimal  user  interfaee 
regarding  medieal  data.  In  FMSS,  you  must  transeribe  medieal  info 
from  treatment  reeords.  FMSS  may  be  useful  in  lower  eehelon 
MTFs  for  loeal  trend  analysis. 
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6.0  Conclusions 

6.1  Functional  Claims  by  Developer 

The  results  indieate  that  the  FMSS  applieation  Version  1.0  generally  worked  as 
advertised  by  the  developers.  The  produet  exeeuted  the  basie  funetions  of 
reeeiving  patient  information  direetly  from  providers;  determining  incidenee 
rates;  generating  trend  analysis  reports;  profding  by  person,  time,  or  plaee; 
traeking  modes  of  disease  transmission;  generating  graphs  and  reports;  and 
providing  aeeess  to  medieal  referenees.  See  Seetion  5.0  and  Appendix  D  for  a 
eomplete  aeeount  of  funetions  and  speeifie  eomments  on  their  performanee. 

6.2  Usability  and  Review  by  Technical  Consultants 

The  program  was  easy  to  use  and  eould  be  learned  rapidly  by  new  users.  The 
usability  and  reviews  by  teehnieal  eonsultants  indieated  that  the  main  area  of 
concern  was  in  access  help  to  support  user  questions  or  correct  user  mistakes. 
However,  there  were  several  cases  in  which  functions  were  not  listed  in  the  user’s 
manual  or  were  described  in  the  user’s  manual  in  a  way  that  was  inconsistent  with 
their  actual  operation.  See  Section  5.0  for  specific  examples. 

6.3  Compatibility  With  Code  23  Program  Architecture 

FMSS  Version  1.0  does  not  appear  to  meet  program  requirements  in  that  it  does 
not  collect  a  complete  set  of  information  on  ICD-9  codes  linked  to  any  input  of 
patient  information.  Also,  it  appears  that  FMSS  Version  1.0  does  not  have  the 
capability  to  export  its  patient  database  with  ICD-9  codes  to  other  applications. 
This  is  important  if  the  medical  records  are  to  be  downloaded  and  analyzed  by 
other  applications,  such  as  MDSS  or  the  Joint  Medical  Work  Station  (JMeWS). 
FMSS  overlapped  with  MDSS  extensively  but  lacked  certain  reports  such  as 
disease  and  nonbattle  injury  (DNBI)  and  provided  less  capability  than  MDSS  for 
analyzing  trends  for  disease  outbreaks.  FMSS  did  provide  some  decision  aid 
information  and  medical  reference  material  that  is  not  found  in  MDSS. 

6.4  Stand-Alone  Capability 

This  review  indicates  that  FMSS  could  function  by  itself  to  support  preventive 
medicine  units  in  the  field.  FMSS  generates  systematic  records  of  patient 
diagnosis  and  treatment,  as  well  as  surveillance  of  disease  trends.  The  system  also 
provides  decision  support  for  diagnosis,  given  provider  location  in  remote 
locations. 

Further  field  testing  will  be  needed  to  validate  provider  use  of  this  application  as  a 
stand-alone  medical  support  system.  The  technical  consultants  in  this  study 
suggested  that  FMSS  had  the  potential  to  support  field  medical  care,  but  two  of 
the  three  were  unsure  it  would  help  them  do  their  jobs  as  designed  and  given 
current  medical  business  practice. 
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There  was  also  eoneern  that  FMSS  was  limited  in  surveillanee  eapabilities 
eompared  with  systems  sueh  as  GEMS  and  MDSS.  However,  FMSS  appears  to  be 
more  useful  for  preventive  medieine  offieers  or  Corpsmen  for  loeal  support  of 
smaller  units  in  remote  loeations,  rather  than  the  serviee-wide  surveillanee  for 
whieh  MDSS  appears  to  be  targeted.  Providers  ean  input  patient  data  direetly  into 
FMSS,  and  MDSS  does  not  have  this  eapability.  In  eontrast  to  MDSS,  FMSS 
requires  only  a  portable  laptop  system;  sueh  a  system  might  be  suitable  for  many 
forward  levels  of  eare,  sueh  as  small  surfaee  ships  and  some  field  medieal  units 
supporting  far-forward  eombat. 
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7.0  Recommendations 

The  present  data  suggest  this  system  has  substantial  potential  and  would  benefit  from 
field  evaluations.  The  developers  might  follow  up  to  request  formal  feedbaek  from 
eurrent  users  in  the  field  to  support  deeisions  on  how  to  field  this  system  in  the  future. 
Other  reeommendations  inelude  the  following: 

•  The  eapability  of  FMSS  to  import  and  export  ICD-9  eodes  needs  to  be  fully 
tested.  This  would  allow  system  integration  with  other  applieations  used  at 
various  levels  of  eare.  This  is  not  addressed  in  the  user’s  manual  or  the 
developers’  advertised  elaims.  In  order  for  FMSS  to  be  compatible  with  other 
medical  surveillance  systems  such  as  MDSS,  a  systematic  set  of  ICD-9  codes  is 
required,  and  they  should  be  exportable  as  files  to  other  applications  such  as 
JMeWS  or  MDSS.  Similarly,  compatibility  may  also  require  that  diagnoses 
reached  via  other  options  in  FMSS  (e.g.,  through  GIDEON  or  by  signs  and 
symptoms)  be  categorized  by  ICD-9  code. 

•  One  technical  consultant  recommended  that  the  Merck  Manual  be  added  as  a 
reference. 

•  Either  the  program  or  the  user’s  manual  should  be  changed  so  that  the 
Surveillance  Graph  functions  as  described,  and  a  section  on  the  Spreadsheet  View 
and  its  features  should  be  included. 

•  One  technical  consultant  suggested  that  it  would  be  useful  to  be  able  to  examine 
outcomes  by  more  than  two  variables. 

•  Some  specific  attention  to  access  to  help  functions  would  increase  usability  of  this 
product.  The  user’s  manual  should  be  updated  to  be  consistent  with  all  application 
functions.  Eor  instance,  time  series  reports  are  described  in  the  user’s  manual  but 
do  not  appear  functional  in  the  current  application. 

•  The  graphics  functions  could  be  enhanced  to  allow  flexibility  for  users  to  analyze 
information.  Options  to  express  surveillance  information  as  rates  should  be 
flexible  and  consistent  with  epidemiological/military  standards. 

•  Some  additional  required  surveillance  reports  such  as  DNBI  could  be  added  to 
this  application.  The  developers  also  need  to  include  the  time  series  and  incidence 
rate  graph  capability  described  as  an  option  by  the  user’s  manual. 
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Readiness  Assessment 


PRODUCT  INFORMATION 

EVALUATION 

Product:  FMSS 

FUNCTIONAL 

Claim 

Requirement 

Observed 

Developer;  NHRC,  Code  23:  Martin  White  PI 

Determines  disease 
incidence  rates/trends 

DoD  medical 
surveillance 

Determined  Incidence 
rates 

Generated  trend 
analysis  reports 

Cost:  $  Not  Determined 

Tracks  modes  of  disease 
transmission 

DoD  medical 
surveillance 

T racked  modes  of 
disease  transmission 

Profiles  patient  by 
person,  time  period,  or 
place 

Electronic  medical 
patient  encounter 
module 

Profiled  patient  by  time 
period  or  place 

Installation/Support 

Software:  Application/Manual  on  CD 

Documentation;  Tech  Report  “A  Field  Medical 

Surveillance  System  for  Deployed 

Forces” 

Training;  None  -  only  user’s  manual 

TECHNICAL 

Claim 

Requirement 

Observed 

Medical  reference 

Medical  reference  and 
decision  support  for  field 

Incorporated  GIDEON 
database 

Specifications;  Windows  NT  4.0  or  2000/MS  Access 

on  laptop  computer 

Generate  graphs  and 
reports 

Epidemiological  analysis 
tools 

Graphics  functional  with 
some  limitations 

64  MB  RAM 

Provide  access  to 
medical  references 

Medical  decision  aids 

Multiple  medical  guides 
available 

CLAIMS 

Functional:  S®®  functional  claims  on  right. 

Technical:  S®®  technical  claims  on  right. 

Compatibility  With  Program  Architecture: 

Not  satisfactory  for  data  transfer  to  JMeWS/MDSS  without  complete  set  of  exportable  ICD-9 
codes. 

Requirements 

Functional:  surveillance,  medical  record, 
medical  reference. 

Rating 

Functional: 

Satisfactory  for  use  in  field  by  PMOs  and  Corpsmen  with  laptop 
computer. 

Technical:  See  technical  claims  on  right. 

Technical: 

Not  satisfactory  for  integration:  ICD-9  codes  must  be  complete  and 
exportable.  Help  functions  and  graphics  could  be  improved. 

Overall: 

Satisfactory  as  stand-alone  tool  for  field  use. 

Not  satisfactory  for  integration  with  other  applications  without  ICD-9 
exporting  capability. 
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Instructions: 

A)  Rate  each  statement  below  as  1-5  for  agreement. 

(1  =  strongly  disagree,  2  =  disagree,  3  =  undecided,  4  =  agree,  5  =  strongly  agree.) 

B)  If  you  have  any  comments,  write  them  in  the  space  provided.  Please  include  any 
suggestions  you  have  and  any  examples  that  support  your  rating. 

1)  Uses  simple  language. 


2)  Language  is  familiar  to  target  user. 


3)  Use  requires  minimal  memory  load. 


4)  Consistent  use  of  key  terms  and  format  for  navigation. 


5)  Feedback:  System  provides  informative  feedback  to 
user  on  actions  performed. 


6)  Navigational  functions  are  easy  to  find  on  every  screen. 


7)  Shortcuts  are  available  for  experienced  or  expert  users  but  are 
not  necessary.  (Accessibility  to  varied  user  expertise.) 


8)  Error  messages  are  clear  and  provide  constructive  solutions. 


9)  Design  is  built  to  prevent  or  minimize  user  error  (forgiveness). 


10)  Provides  help  if  necessary. 
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1 1)  Aesthetics:  Graphic  design  is  simple  and  intuitive. 


12)  Flexibility:  User  can  do  what  they  want  when  they  want. 
User  doesn’t  have  to  go  through  unwanted  steps  every  time. 


13)  Easy  to  learn  to  use.  User  can  understand  basic  functions  within  an  hour. 


14)  Basic  functions  are  grouped/organized  in  a  reasonable  fashion. 


15)  User  can  undo  or  redo  actions. 
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Your  Name 

Phone  E-mail  address 

Date 

Gender:  Male  Female 

Active  Duty:  Yes  No 

Service: 

Navy 
Army 
Air  Force 
Marines 

What  was  your  position  title? 

GMO  (General  Medical  Officer) 

PMO  (Preventive  Medicine  Officer) 

EHO  (Environmental  Health  Officer) 

IDC  (Independent  Duty  Corpsman) 

Nurse 

Physician’s  Assistant 
Command  Surgeon 
Task  Eorce  Surgeon 
CINC  Surgeon 
Epidemiologist 
Task  Eorce  Commander 
HAZMAT 
Other 

Which  area  best  describes  your  education  and/or  training?  (Choose  one.) 

Statistics 
Epidemiology 
Preventive  Medicine 
Other 

How  much  experience  did  you  have  with  medical  data  software?  (months/years) 


How  much  experience  did  you  have  working  aboard  a  deployed  ship?  (months/years) 
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Your  written  comments  will  assist  in  the  development  of  this 
technology  for  the  medical  support  of  the  armed  forces. 
Please  provide  them  wherever  possible,  even  if  it’s  just  a  few 
key  words. 


FMSS  Information 


1.  The  medical  information  provided  through  FMSS  was  useful. 

□  Strongly  Agree  □  Agree  □  Neither  Agree  Nor  Disagree  □  Disagree  □  Strongly  Disagree 

□  Not  Observed 

Please  Explain:  _ 


2.  The  medical  information  provided  through  FMSS  was  easy  to  use. 

□  Strongly  Agree  □  Agree  □  Neither  Agree  Nor  Disagree  □  Disagree  □  Strongly  Disagree 

□  Not  Observed 

Please  Explain:  _ 


3.  The  medical  information  FMSS  provided  was  presented  in  a  useful  format. 

□  Strongly  Agree  □  Agree  □  Neither  Agree  Nor  Disagree  □  Disagree  □  Strongly  Disagree 

□  Not  Observed 

Please  Explain:  _ 


4.  The  quality  of  the  medical  information  provided  by  FMSS  is  better  than  that  provided  by 
previous  reporting  methods.  (Please  specify  reporting  methods  previously  used.) 

□  Strongly  Agree  □  Agree  □  Neither  Agree  Nor  Disagree  □  Disagree  □  Strongly  Disagree 

□  Not  Observed 

Please  Explain:  _ 
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5.  The  FMSS  system  was  flexible  enough  to  meet  my  needs.  I  could  set  it  up  to  do  what  I  wanted 
it  to  do  efficiently. 

□  Strongly  Agree  □  Agree  □  Neither  Agree  Nor  Disagree  □  Disagree  □  Strongly  Disagree 

□  Not  Observed 

Please  Explain:  _ 


6.  FMSS  would  help  me  do  my  job. 

□  Strongly  Agree  □  Agree  □  Neither  Agree  Nor  Disagree  □  Disagree  □  Strongly  Disagree 

□  Not  Observed 

Please  Explain:  _ 


7.  The  most  useful  thing  about  FMSS  for  my  job  would  be 


8.  The  least  useful  thing  about  FMSS  for  my  job  was 


9.  If  I  could  add  one  thing  to  FMSS  it  would  be 


10.  What  features  of  FMSS  did  you  like? 
Please  Explain:  _ 


11.  What  features  of  FMSS  did  you  dislike? 
Please  Explain:  _ 


12.  Where  would  you  consider  FMSS  a  best  fit  in  the  military  medical  arena? 
Please  Explain:  _ 
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FMSS  TEST  LOG 

PROJECT  ID:  Field  Medical  Surveillance  System 

DATE:  July  2003 

DEVELOPER/TEST  ENGINEER: 

PROGRAM/MODULE:  Field  Medical  Surveillance  System 

TEST 

CASE  ID 

TEST 

CONDITIONS 

TEST 

RESULTS 

PASS/FAIL 

COMMENTS 

PROBLEMS  ENCOUNTERED 

FMSS-F01 

pass 

User  was  able  to  create  a 
new  database  or  open  an 
existing  database  without 
difficulty. 

FMSS  -  F02 

pass 

User  was  able  to  verify 
existing  units  or  unit  and 
was  able  to  record  basic 
demographics  for  each 
patient  without  difficulty. 

FMSS  -  F03 

pass 

User  was  able  to  enter 
diagnosis/treatments  and 
edit  diagnosis  for  each 
patient  without  difficulty. 

Page  13  of  the  user’s  manual  displays  a  screen  shot 
of  the  FMSS  software.  The  screen  shot  displays  a 
Provider  drop-down  box  under  the  tab  titled 

Diagnosis;  however,  the  software  displays  a  Visit 
drop-down  list.  Page  14  of  the  user’s  manual  states 
that  the  software  “also  allows  you  to  select  a  provider 
(Corpsman,  Doctor,  Specialist,  or  Other).’’ 

Recommend  changing  the  user’s  manual  to  reflect 
actual  software  screens.  The  Provider  drop-down  list 
displays  when  the  tabs  titled  Consultations,  Clinical 
Studies,  and  Treatments  are  chosen. 

FMSS  -  F04 

pass 

User  was  able  to  query 
the  database  to  view  the 
different  reports  and 
graphs. 

Trend  Analysis  is  not  listed  in  the  user’s  manual  but  is 
offered  as  a  report. 
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FMSS  -  F05 

pass 

User  was  able  to  access 
and  view/ review 
resource/reference 
manuals. 

FMSS  -  F06 

pass 

User  was  able  to  access 
the  online  help. 

The  online  help  feature  “About  GIDEON”  and  “About 
AFMIC/MEDIC”  displays  a  window  but  this  window 
cannot  be  closed  using  the  typical  close  function 
seen  upper  right  marked  X;  it  appears  to  be  disabled. 

A  user  has  to  close  the  feature  by  clicking  on  the 
display.  This  is  not  intuitive. 

FMSS  -  F07 

pass 

User  was  able  to  open 
an  existing  database  with 
ease  from  either  the  main 
screen  or  the  File  menu. 

Documentation  in  the  user’s  manual  is  inconsistent 
with  the  actual  screen.  Page  5  of  the  user’s  manual 
states  that  you  can  open  an  existing  database  either 
at  the  opening  screen  or  from  the  File  menu.  The 
user’s  manual  states,  “when  you  first  open  FMSS,  a 
screen  will  appear  asking  if  you  want  to  open  an 
existing  database.”  However,  the  installed  system 
does  not  provide  an  opening  screen;  instead,  the 
system  opens  to  the  last  database  that  was  viewed. 

This  was  found  on  both  a  client  and  stand-alone 
system. 

FMSS  -  F08 

pass 

User  was  able  to  create  a 
new  unit  from  the  File 
menu  with  ease. 

FMSS  -  F09 

pass 

User  was  able  to 
open/review  reports  from 
the  Reports  menu. 

FMSS-FO10 

fail 

User  was  able  to  create  a 
new  demographic  after 
resolving  one  problem. 

FMSS  will  allow  a  user  to  type  in  a  social  security 
number  with  10  digits,  but  will  not  properly  space  the 
digits  if  they  are  typed  too  quickly. 

FMSS-F011 

pass 

User  was  able  to  edit  a 
demographic. 

When  editing  patient  demographics,  if  a  date  within 
the  date  of  birth  is  a  single-digit  number,  the  system 
will  prompt  the  user  to  re-check  and  re-enter  the  DOB 
to  match  the  format.  When  the  file  was  created,  the 
system  would  not  allow  the  incorrect  date  format  to 
be  entered,  so  FMSS  is  changing  the  format 
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FMSS-F012 


FMSS-F013 


FMSS-F014 


FMSS-F015 


FMSS-F016 


FMSS-F017 


FMSS-F018 


FMSS-F019 


FMSS  -  FO20 


FMSS-F021 


FMSS  -  F022 


pass 

User  was  able  to  delete  a 
demographic. 

pass 

User  was  able  to  add  a 
diagnosis. 

pass 

User  was  able  to  use  the 
new  Diagnosis  box  to 
select  a  provider  and 
enter  consultations. 

pass 

User  was  able  to  enter  a 
diagnosis  for  a  patient  or 
patients. 

pass 

User  was  able  to  enter  a 
diagnosis  for  a  patient  or 
patients  from  signs  and 
symptoms. 

pass 

User  was  able  to  profile 
the  differential  list  of 
diseases. 

pass 

User  was  successful  at 
reviewing  information  on 
epidemiological  data. 

pass 

User  was  successful 
viewing  what  countries 
the  disease  is  known  in. 

pass 

User  was  successful  in 
selecting  diagnosis  using 
the  disease  profiles. 

pass 

User  was  successful  in 
selecting  a  diagnosis 
using  GIDEON. 

pass 

User  was  successful  in 
selecting  and  entering 
consultations  on  patients 
or  patient. 
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FMSS  -  F023 

pass 

User  was  successful  in 
selecting  and  entering 
clinical  studies. 

FMSS  -  F024 

pass 

User  was  successful  in 
selecting  and  entering 
treatments. 

FMSS  -  F025 

pass 

User  was  successful  in 
selecting  and  entering 
duty  status  on  patients  or 
patient. 

FMSS  -  F026 

pass 

User  was  successful  in 
selecting  and  entering 
diagnosis  on  patients  or 
patient. 

FMSS  -  F027 

pass 

User  was  successful  in 
selecting  and  deleting 
diagnosis  on  patients  or 
patient. 

FMSS  -  F028 

pass 

User  was  able  to 
view/print  the  daily  log  for 
a  selected  day. 

FMSS  -  F029 

pass 

User  was  able  to 
view/print  a  patient’s 
history. 

FMSS  -  FO30 

pass 

User  was  able  to 
view/print  the  reportable 
conditions. 

FMSS-F031 

pass 

User  was  able  to 
view/print  the 

Classification  Report. 
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FMSS  -  F032 

fail 

User  was  not  able  to 
report  on  unit(s)  patient 
variables  to  one  or  more 
diseases  based  on  the 
number  of  cases  or  the 
incident  rate. 

User  was  unable  to  create  or  view  a  Time  Series  or  an 
Incidence  Rate  Series  Graph.  There  was  no  option  to 
do  so,  even  though  it  is  listed  in  the  user’s  manual. 

The  user  can  pull  up  a  text  report,  but  no  graphs  are 
available. 

FMSS  -  F033 

pass 

User  was  able  to  graph 
unit(s)  patient  variables 
to  one  or  more  diseases 
based  upon  the  number 
of  cases  or  the  incident 
rate. 

There  is  no  option  to  specifically  choose  X  and  Y  axis. 
This  action  is  not  user-friendly.  Recommendation 
would  be  to  identify  the  X  and  Y  axis  in  the  Text  Edit 
box  identifying  the  X  and  Y  axis  of  the  selected 
variable.  Also,  if  the  Y-axis  is  removed,  it  should  not 
automatically  move  to  the  X-axis  position. 

FMSS  -  F034 

pass 

User  was  able  to 
access/view  the  online 
re  so  u  rce/ref  e  ren  ce 
manuals. 

FMSS  -  F035 

pass 

User  was  successful  in 
selecting  and  viewing  the 
trend  analysis  report. 

Not  listed  in  user’s  manual. 
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Problem  Description  #1 

In  the  Spreadsheet  View,  the  user  is  able  to  modify  data  eells  and  save  ehanges  to  Unit, 
Demographic,  and  Encounter  data.  Changes  made  in  this  way  may  not  be  reflected  when  the  data 
are  viewed  through  other  features.  For  example,  changing  and  saving  an  entry  under  Race  in  the 
spreadsheet  cell  did  not  result  in  a  change  when  the  same  patient’s  record  was  viewed  through 
the  Edit  Demographics  menu. 

How  to  Repeat 

In  Spreadsheet  View  (under  the  File  menu),  click  on  any  cell  in  any  of  the  three  spreadsheets. 
Edit  the  data  within  the  cell  and  click  on  any  cell  in  the  next  row  up  or  down.  Click  OK.  The 
change  will  appear  when  the  spreadsheet  view  is  recalled,  but  will  not  appear  when  the  patient 
data  are  requested  through  the  Unit,  Demographic,  or  Encounter  Edit  menus. 

Problem  Description  #2 

FMSS  allows  the  user  to  create  and  define  new  units  within  a  database.  When  a  new  unit  was 
created  and  viewed  under  Spreadsheet  View,  the  Encounters  page  of  the  spreadsheet  contained  a 
list  of  encounters,  even  though  no  data  for  any  patients  had  been  entered.  The  listed  encounters 
could  not  be  accessed  for  editing  or  deletion.  This  problem  only  appears  in  the  Middle  East.fpd 
file,  and  appears  to  be  a  database  bug. 

How  to  Repeat 

Open  the  Middle  East.fpd  database  through  Open  Database  under  the  File  menu.  Select  Create 
Unit  from  the  File  menu  and  create  a  unit  using  any  data.  Select  Spreadsheet  View  from  the  File 
menu  and  select  the  newly  created  unit  from  the  Unit  spreadsheet.  The  Demographics 
spreadsheet  will  be  empty,  but  the  Encounter  spreadsheet  will  have  several  entries  in  it.  The  New 
and  Edit  buttons  on  this  spreadsheet  are  inactive. 
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Acronyms 

AFMIC  Armed  Forces  Medical  Intelligence  Center 

CCDM  Control  of  Communicable  Diseases  Manual 

CHCS  Composite  Health  Care  System 

CINC  Commander-in-Chief 

DNBI  Disease  and  Nonbattle  Injury 

DOD  Department  of  Defense 

EHO  Environmental  Health  Officer 

EMSS  Eield  Medical  Surveillance  System 

EMT  Eield  Medical  Technologies 

GEMS  Global  Expeditionary  Medical  System 

GIDEON  Global  Infectious  Disease  and  Epidemiology  Network 

ICD-9  International  Classification  of  Diseases,  9th  Revision 

JMeWS  Joint  Medical  Workstation 

MDSS  Medical  Data  Surveillance  System 

MEDIC  Medical  Environmental  Disease  Intelligence  and  Countermeasures 

MTE  Medical  Treatment  Eacility 

NHRC  Naval  Health  Research  Center 

PMO  Preventive  Medicine  Officer 

SAMS  SNAP  Automated  Medical  System 

SOP  Special  Operations  Porces 

SOMDS  Special  Operations  Medical  Diagnostic  System 

T&E  Test  and  Evaluation 

TSRE  Tri-Service  Reportable  Events  Guidelines  and  Case  Definitions 
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The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data 
sources,  gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other 
aspect  of  this  collection  of  information,  including  suggestions  for  reducing  the  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and 
Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302,  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person 
shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  Control  number.  PLEASE  DO  NOT  RETURN 
YOUR  FORM  TO  THE  ABOVE  ADDRESS. 
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